Skip to content

fix: release query resources and drain Store RPCs - #3294

Closed
imbajin wants to merge 31 commits into
apache:masterfrom
hugegraph:task/topling-split-lifecycle
Closed

imbajin wants to merge 31 commits into
apache:masterfrom
hugegraph:task/topling-split-lifecycle

Conversation

@imbajin

@imbajin imbajin commented Oct 8, 2026

Copy link
Copy Markdown
Member

Requests can leave scan work or native resources alive after cancellation, and Store can close its engine before in-flight RPCs finish. This change releases request-owned resources on failure/cancellation and drains Store RPC work before storage shutdown.

  • Release every transaction owner and preserve cleanup failures.
  • Close scan iterators and child streams when requests fail or are cancelled.
  • Stop accepting new Store work, finish or cancel in-flight work, then close storage.

Based on current Apache master, including the merged ToplingDB integration. This is a general lifecycle fix for Server/Store; it does not add a provider or change how users switch to ToplingDB. Changes already merged in apache/hugegraph#3275 are retained rather than duplicated.

Validation: Java 17 full-repository compilation and focused transaction, scan, cancellation, shutdown and legacy serialization regressions pass. Existing review threads are resolved; CI for this updated commit remains to be confirmed. See docs/transaction-lifecycle.md for lifecycle behavior.

Doc - Done.

imbajin added 30 commits October 3, 2026 22:37
- release request and task transactions across graphs
- protect scan callbacks until Store shutdown completes
- cover lifecycle boundaries with regression tests
- document shutdown behavior and timeout diagnosis
- keep caller transactions open when traversers close
- rollback failed worker transactions and reset thread state
- run Store shutdown tests through the active Maven suite
- cover task commit and cleanup with real RocksDB tests
- require the central server profile and test report
- verify every selected Store profile has a session
- retain strict execution order and coverage checks
- reject interrupted scans without partial success
- wake closed channels without interrupting RPC callers
- verify cancellation status through active scan paths
- retain library and executable Node artifacts
- expose stop diagnostics from smoke cleanup
- preserve failure status and retained PID evidence
- retain existing native shutdown timeout
- include the merged runtime alignment
- retain lifecycle changes as a separate PR
- inherit current upstream CI configuration
- preserve both upgrade and lifecycle documentation
- keep the ordinary code margin at 120 columns
- allow Markdown paragraphs without hard wrapping
- show a 160-column Markdown visual guide
- align CodeRabbit instructions with repository style
- separate response completion from resource release
- isolate cancellation callback failures
- preserve native iterator close failures
- transfer iterator ownership before prefetch
- cover Spring shutdown and real JNI boundaries
- Keep the newest run for each workflow and PR
- Verify source freshness before automatic retries
- Test retry decisions and document the CI policy
- Keep one active run per workflow and PR
- Preserve push and manual workflow executions
- Cover CI workflows on this PR target branch
- Reserve the exec classifier for distribution assembly
- Avoid compile-time resolution of packaged artifacts
- Keep the existing assembly filename and runtime inclusion
- separate natural completion from cancellation
- retain failed cleanup owners until retry succeeds
- close nested scan iterators and serialize receipts
- verify cancellation and ordered scan backpressure
- deliver credit-backed results after request half-close
- cancel client streams when iterators close early
- preserve queued final results across completion races
- verify matched client and server stream behavior
- report outstanding scan count at the wait interval
- retain details only for actual cleanup failures
- preserve shutdown waits and interrupt handling
- preserve cancellation precedence in terminal callbacks
- finish cancelled scans after automatic iterator cleanup
- cover blocked close with callback and unary gRPC tests
- document cancellation and retain native drain ownership
- capture the observed wait state before asserting
- retain database and failed-cleanup ownership checks
- verify scan and aggregate shutdown regressions
- adopt the Java 17 baseline
- retain Gremlin and lifecycle semantics
- preserve both test-suite additions
- propagate transport cancellation independently
- reject repeated queries before iterator allocation
- cover real cancellation and single terminal callbacks
- document request and response lifecycle boundaries
- wait until response callback is in flight
- trigger cleanup failure in the intended order
- preserve worker ownership and terminal assertions
- close write batch and options at final detach
- retain owners while another lease is active
- guarantee both disposal attempts and closed state
- cover real JNI owners across repeated worker requests
- select supported auth fixtures by actual graph backend
- run cleanup assertions without a backend system property
- give the OLAP sentinel fixture a valid vertex ID
- assert the intended schema rejection source
- retain schema identity across request cleanup
- keep cache invalidation listeners until graph close
- dispose only the matching graph listener generation
- cover schema append and API update regressions
- observe normal sender completion before teardown
- avoid inline transport cancellation during completion
- retain early cancel and resource release assertions
- check published terminal state before timed poll
- preserve final-batch queue visibility ordering
- cover exhausted single and merged streams
- register the regression in the client suite
- detach owned store listeners without a cache holder
- preserve reopened listener generation ownership
- clear auth projects using the actual graph backend
- retain reopen failures and clean temporary data
- propagate transaction cleanup failures after all owners close
- retain graph-scoped schema cache listeners
- cancel all started result streams on close
- register graph scans for deterministic shutdown cleanup
- retain merged Topling and shared schema behavior
- drain Store streams before storage shutdown
- preserve cleanup failures and update regressions
- exclude HTTP libraries from Node test dependency
- preserve production dependencies and security gate
- verify Store lifecycle tests without web jars
@codecov

codecov Bot commented Oct 8, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 74.62687% with 17 lines in your changes missing coverage. Please review.
✅ Project coverage is 41.62%. Comparing base (d00efce) to head (a73b864).

Files with missing lines Patch % Lines
...n/java/org/apache/hugegraph/StandardHugeGraph.java 40.00% 7 Missing and 2 partials ⚠️
...aph/store/client/query/CommonKvStreamObserver.java 75.00% 2 Missing and 2 partials ⚠️
...he/hugegraph/backend/store/BackendSessionPool.java 80.00% 2 Missing ⚠️
.../apache/hugegraph/backend/tx/GraphTransaction.java 0.00% 0 Missing and 1 partial ⚠️
...g/apache/hugegraph/task/StandardTaskScheduler.java 0.00% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##             master    #3294      +/-   ##
============================================
+ Coverage     41.42%   41.62%   +0.19%     
- Complexity     6931     6979      +48     
============================================
  Files           762      762              
  Lines         66773    66803      +30     
  Branches       8939     8948       +9     
============================================
+ Hits          27662    27804     +142     
+ Misses        35957    35824     -133     
- Partials       3154     3175      +21     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@imbajin

imbajin commented Oct 9, 2026

Copy link
Copy Markdown
Member Author

Superseded by #3302 (Server transactions), #3303 (Store RPC/Scan) and #3304 (Store-wide shutdown/native drain, dependent on #3303). The original branch and commits are retained. The focused PRs will receive subsequent fixes and validation.

@imbajin imbajin closed this Oct 9, 2026
@imbajin
imbajin deleted the task/topling-split-lifecycle branch October 10, 2026 10:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant